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ABSTRACT 


The advent of convolutional codes provides us with extremely 
low error rates at the expense of data rejection. These codes are 
compared to a perfect code and to a decision system capable of 
variable deletion rates. The deletion rates can be adjusted to val- 
ues equal to those of convolutional codes resulting thus in a fair 
comparison. 
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A COMPARISON OF DECISION SYSTEMS 
WITH DATA REJECTION 


1. INTRODUCTION 

It has been shown (References 1 and 2), that convolutional codes (References 3 and 4) can re- 
duce error rates to extremely low values. The only drawback is the necessity to reject data. This 
deletion rate, P del , is a function of E/N 0 and the resulting probability of error. The above results 
are shown in Figure 1. Unfortunately this type of graph does not offer a fair comparison between 
the convolutional codes and the algebraic ones, or other detection systems due to the fact that the 
curves representing the algebraic codes do not suffer from any rejection of the data. For example 
in the case of the biorthogonal n = 8 we see that for E/N 0 = 4 db 

P e = 7. 214 x 10 -4 and P c = 1 - P e = .9992786 


On the other hand for the convolutional code and the same 
signal to noise ratio 

P e = 3 x 10 -4 and P del = .03 

thus 

P c = 1 - P e - Pdel = 9697 

The above simple calculation shows that although the 
convolutional code offers a smaller probability of error, 
the algebraic code offers a higher probability of correct 
detection. 

The performance of one code over another can be 
described by the average risk function B(P e , P c , C, , c 2 , 
C 3 ) where c,, c 2 , C 3 are cost functions, and Bean be 
simply expressed as 

B = Cj P e + C 2 P + C 3 P deletion 



NORMALIZED SIGNAL TO 
NOISE RATIO, E/N 0 (DB) 


= c,p e + C 2 p c + c 3 (1-P c -P e ) 


1 


Figure 1 —Comparative Performance of 
Communication Systems. 



Thus unless C 15 C 2 and C 3 are known one cannot pass a judgment on the relative performance of 
two systems. However if one of either P e , P c , or p de i et ion cou ^ m ^de the same for the two sys- 
tems in question (of course under equivalent E/N 0 ), then a comparison is possible without the 
knowledge of the c/s. 

For example if the probability of deletion (D) is made the same for both systems then the two 
b t s become 


B, 

= CjP., + C 2 P cl 

+ c 3 d 


b 2 

^ L^e2 + ^2 ^c2 

+ c 3 d 


B, - B 2 

= C . ( P el - P e 2 ) + 

C 2 ( p c. -P e2 ) 

(1) 


but 


1 - P . - D 


( 2 ) 


Substituting (2) in (1) we get 

B, - B 2 = C,(P el -P. a ) + C 2 (1-P el -1+P. a ) 

= ( c ,-c 2 ) (P.1-P, 2 ) • 


Since C, -c 2 > 0* then system 2 is better than system 1 if P e2 <P el . Thus we have showed that one 
way to compare the two systems is to keep E/N 0 and D the same for both systems, and then examine 
P e for each of the systems; the system with smallest P e gives a better performance. 

There are at least two ways to lower the probability of error of a system and simultaneously 
suffer deletions. This error to deletion tradeoff can be accomplished by changing from a strictly 
error correcting code, (let us say it corrects up to and including e x errors), to a combination er- 
ror correction and detection code (say it corrects up to and including e 2 errors and detects up to 
and including d errors). In general for the same redundancy the sum of the number of the errors 
corrected and the errors detected for system 2 is larger than the number of errors corrected in 
system 1, or 

e 2 < e i < e 2 + d i ■ 

*C2 < for all cases where the error is undesirable and correct detection is desirable, i.e., the error cost is larger than the correct 
detection cost. 
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The above equation indicates that although system 2 will correct less errors than system 1, sys- 
tem 2 is able to reject data if errors are detected and, since in general e 2 + d > e t , the number of 
undetected and uncorrected errors is less than that of system 1. Consequently the error rate in 
system 2 is smaller than that of system 1. The second method is a specific case of the more gen- 
eral case discussed in Reference 5. This technique is much simpler than error detection and 
correction. Under this technique the largest of the correlator outputs is detected. Up to this point 
the procedure is identical to the Maximum Likelihood Decision Scheme. The value of the ratio 


Bi 



a 


is measured and the data is rejected if this value is smaller than a predetermined threshold B t . 
If the value of the Statistic is larger than B t the decision is that x (N) is the useful signal. 


2. PERFORMANCE OF THE GOLAY (23, 12) CODE WITH ERROR CONTROL 

This code was chosen because it is a perfect code and will correct all three or less errors. 
The word error probability is given by 


p w = 2Z w pi(i-p)n ’ i 

i = 4 


where P is the bit error probability and is given by 


iya-p) e/n 0 


; -t 2 /2 


dt 


(3) 


(4) 


E/N 0 is the Energy per bit to Noise power per unit bandwidth. For antipodal signals 1 - p = 2; if 
E/N 0 is the energy per information bit to N 0 then the energy per bit to N 0 is k/n (E/N 0 ) because there 
are k information and n bits per word. Thus for the Golay (23, 12) Code 


P 



[ 


Y 24E/ 23N q 


.-t 2 /2 


dt 


(5) 


is the largest output, fj, is the sample mean and a is the sample standard deviation. 
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WORD ERROR PROBABILITY 


(3) and (5) were used to plot the performance curve of this code (shown as the dotted line) in Fig- 
ure 2. Equation 4 is a special case of the generalized equation. 



t 


( 6 ) 



1 2 3 4 5 6 7 8 9 10 11 12 13 

NORMALIZED SIGNAL TO t = x//2 

NOISE RATIO, E/N 0 (DB) 


Figure 2 — Performance of the 
Golay (23, 12, 3) code. 



then 
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This is clearly the same as Equation (2). Equation (4) gives the probability of error under a Max- 
imum Likelihood Decision provided the largest signal exceeds a given threshold B t . If it does not 
exceed B t no decision is made (the data is erased). Under this decision scheme one can reduce 
the probability of error to any desired value at the expense of higher rejection rates. 

The probability of correct detection for binary antipodal signals is given by 


Q 



e -(s-A) 2 A 


ds 



e - * 2 / 2 dx 


with A = i 4E/N 0 . P e is given by (4) and the erasure rate E b is 


= l - P - P. 


(7) 


( 8 ) 


This detection scheme can be added to coding in the following manner. 

The n coded bits are detected according to this scheme withP e , P c , and E as given by Equa- 
tions (6), (7), and (8). 

If the number of erasures in the word of n bits is larger than r we reject the data. 

Then the word deletion rate (from now on referred to as deletion rate) is given by 


D(E b> r) 


£ (?) v (■-■o' 

j=r+l 



(9) 


under this scheme the word error probability for an (n, k, e) perfect code* with the erasures as- 
sumed as errors is 


p; = £] £ (") ( n I l ) k p j ^ ( 10 ) 

i = 0 j — e + 1— i 

where P = P e and Q = P c = 1 -P. Equation (10) is an upperbound on the word error probability be- 
cause under this scheme all erasures are assumed to be errors. To find the exact word error 
probability let us establish a decision rule as follows. If the number of bits that do not satisfy the 


*(n, k, e) -* n bits per word; k information bits; corrects all e or less errors. 
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threshold conditions* is larger than a specified number r, the word is deleted. If it is less than 
r then a Maximum Likelihood decision is made on all n bits. Under this decision rule the deletion 
rate is still given by Equation (9). 

The expression for the word error probability is found in this manner. There are two types 
of bits in the n-bit word. Type-one bits satisfy the threshold requirements and have P and Q given 
by 


prob (x (N) > B t ; Oea>y 

(ii) 

prob^x (N) > B t ; Oeft- oJj 

(12) 


Type-two bits do not satisfy the threshold requirements. The probability of error for those bits is 

E p = Prob(x (N) <B t ;0€oj) 

= prob (9eu) Vx (N) J - prob (x (N) > B t ; &eoJj 

^MLDS ^ * ' (13) 


Similarly the probability of correct decision for the type-two bits is 

E q “ QmLDS ~ Q 


(14) 


and the rejection rate is 


E + E„ 


(15) 


using Equation 11-15 and 


Qmlds 


* e mlds 


then 


= E + E„ - P, 


MLD S 


- P - Q 


- P ~ Q 


•Those bits were erasures in the previous scheme. 

bi) is the noise parameter space and Cl ~ to is the signal parameter space. 
-f-MLDS stands for Maximum Likelihood Decision System. 
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an obvious result. The word error probability for i type-two correct bits, i type-two incorrect bits, 
j incorrect type-one bits, and correct type-one bits in an e error correcting n-bit code is 

given by 


■ ft 

1=0 i = 0 j=e+l-i 


i — j 


(16) 


with l + i < r . Since from Equation (9) the deletion rate is a function of E^and r, different combina- 
tions of E b and ^will result in the same D; the above results are shown in Figure 3. 

For deletion rates .03 and .3, E^and the corresponding r were read from Figure 3 as shown in 
Tables 1 and 2. 



WORD DELETION RATE (D) 


Figure 3— Erasure rate vs word deletion rate for n = 23. 


Next p(b,) and Q(B t ) were found for all B t for binary antipodal signals by using Equations (6) 
and (7), A = V4(k/m) (E/N 0 ) ; the P and Q corresponding to the required erasure rate E^were found as 
shown on Tables 1 through 5 for the signal to noise ratios of interest; the results were plotted in 
Figure 2 together with the usual performance of the Golay Code (zero erasure rate). An examina- 
tion of Figure 2 indicates that the performance of the convolutional decoding is superior than the 
above scheme of decoding. 
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Table 1 


D = . 

3 E/N 0 

= 3 db 

Qmlds “ *9289 


r 

E 

B, 

P 

Q 

Pw 

1 

.048 

Cn 

CO 

.0608 

.8912 

.0451 

2 

.083 

.79 

.0547 

.8623 

.04423 

3 

.12 

.97 

.0483 

.832 

.04310 

4 

.159 

1.13 

.043 

.798 

.04227 

5 

.198 

1.27 

.038 

.764 

.04208 

6 

.24 

1.40 

.0325 

.7275 

.04103 

7 

.278 

1.51 

.0298 

.6932 

.04258 

8 

CS) 

00 

1.62 

.0248 

.6552 

.04304 

9 

.362 

1.73 

.0212 

.6168 

.04393 

D = . 

03 E/N 0 = 

3.5 db 

Q mlds ~ *9289 


3 

.053 

.58 

.059711 

.887289 

.072 

5 

.1055 

.90 

.050658 

.843842 

.0715 

7 

.170 

1.17 

.04133 

.78867 

.0727 


Table 2 


D = 

05 E/N„ = 

3.5 db 

Qmlds *93962 


r 

E 

B. 

P 

Q 

Pw 

1 

.018 

.26 

.05637 

.92563 

.04184 

3 

.064 

.76 

.048 

.888 

.04153 

5 

.12 

1.08 

.0397 

.8403 

.04177 

7 

.187 

1.31 

.0316 

.7814 

.04245 

9 

.259 

1.56 

.0244 

.7166 

.04453 

11 i 

.337 

1.77 

.01818 

.64782 

.04745 


Table 3 


D = 

03 E/N 0 

= 4 db 

Qmlds ~~ .94962 


r 

E 

B. 

P 

Q 

Pw 

1 

.0135 

.25 

.0475 

.9390 

.0245 

3 

.053 

.78 

.04062 

.90638 

.02441 

5 

.1055 

1.10 

.0334 

.8611 

.02444 

7 

.170 

1.39 

.0263 

.8037 

.0248 

9 

.239 

1.62 

.0203 

.7407 

.0262 

11 

.315 

1.84 

.0150 

.670 

.0282 

D = . 

,3 E/N 0 

, = 4 db 

Qmlds ~ .94962 


1 

.048 

.74 

.041423 

.910577 

.0143 

3 

.012 

1.18 

.031693 

.848307 

.0134 

5 

.198 

1.49 

.023732 

.778268 

.0130 

7 

.278 

1.73 

.0174149 

.704585 

.0134 






Table 4 


D = . 

03 E/N 

o = 6 db 

Qmlds ” *980256 


r 

E 

B t 

P 

Q 

Pw 

3 

.053 

1.31 

.01377 

.93323 

.000848 

5 

.1055 

1.66 

.01026 

.88424 

.000854 

7 

.170 

1.94 

.00729 

.82271 

.000903 

d = 

E/N 0 = 6 db 

Qmlds .980256 


3 

.12 

.172 

i 

.0094945 

.8705055 

.0003462 

5 

.198 

.205 

.006299 

.795701 

.0003458 

7 

.278 

.230 

.00412 

.71788 

.0003897 


Table 5 


D = . 

03 E/N 0 

= 8 db 

Qmlds .995087 


r 

E 

B t 

P 

Q 

Pw 

3 

.053 

2.02 

.002494 

.944506 

.0000032 

5 

.1055 

2.38 

.001566 

.892934 

.0000035 

7 

.170 

2.67 

.000951 

.829049 

.0000042 

D = 

3 E/N 0 

= 8 db 

Qmlds .995087 


3 

.12 

2.43 

.0014555 

.8785445 

.83 x 10‘ 6 

5 

.198 

2.78 

.000772 

.801228 

.99 x 10' 6 

7 

.278 

3.04 , 

.000432 

.721568 

1.44 x 10' 6 


3. ORTHOGONAL CODES WITH DATA REJECTION 

The performance of orthogonal signals when rejection is allowed has already been analyzed 
in Reference 5 for both the coherent and non-coherent cases, and for six different statistics. The 
simplest* statistic is 


B, 


X (N) 


a 


and for ^ = 0 and <x = 1 it becomes the familiar largest of the signals. The decision scheme is 
this. If the largest signal x (N) , is larger than B t the signal is accepted as the useful signal. If 
x (N) is smaller thanB t the data is rejected. Under this scheme the probability of correct decision 
is 

N- 1 

(14) 


*This statistic makes most of the assumptions as to the knowledge of {J. and cr. 


r 

lr 1 

f s (s)ds 

j f(x) dx 


9 




where f s ( z) is the pdf of the useful signal and f(z) is the pdf of the noise signals. The probability 
of error is 


P 


e 


(N-l) 



f s (s) ds 


(15) 


and the rejection rate is 


D - 1 - P c - P« • 


(16) 


It should be noted that with B t = 


and 


D = 0 . 



NORMALIZED SIGNAL TO NOISE RATIO, 
E/N 0 (DB) 

Figure 4— Performance of biorthogona! codes for 
n = 8 and deletion rates of 0, .03, .1, .3 (For 
this large n the performance of orthogonal and 
biorthogonal codes is indistinguishable for all 
practical purposes). 


This last condition on B t results in the Maximum 
Likelihood Decision Scheme. The results of the 
performance of the orthogonal signals are shown 
in Figures 4 to 9. 



NORMALIZED SIGNAL TO NOISE RATIO, 
E/N c (DB) 

Figure 5— Performance of orthogonal and 
biorthogonal n = 4 and D = 0, .03, .1, .3. 
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WORD ERROR PROBABILITY WORD ERROR PROBABILITY 



NORMALIZED SIGNAL TO NOISE RATIO, 
E/N o (DB) 



Figure 6— Performance of orthogonal and Figure 7— Performance of orthogonal and 

biorthogonal n = 5 and D = 0, .03, .1, .3. biorthogona! n = 6 and D = 0, .03, .1, .3. 



Figure 8— Performance of orthogonal and Figure 9— Performance of orthogonal 

biorthogonal n = 7 and D = 0, .03, .1, .3. n = 8 and D = 0, .03, .1, .3. 






4. BIORTHOGONAL CODES WITH DATA REJECTION 


Application of the procedure of the previous paragraph to biorthogonal signals* results in 


-a> s ' 

P c = j f s (s)ds j f(x) dx 


-l(N/2)- 1 


similarly 


where 


P e = a(l-a)t 


a 




|(N/2)-2 


f(t)dt 



f s (s) ds 


and 


D = 1 - P c " Pe ■ 

The results of this type of detection have been plotted in Figures 4 to 9. 


5. CONCLUSIONS 

Figure 2 shows how the performance of the Golay (23, 12, 3) code (dotted line) can be improved 
by allowing deletions for 3% or 30% of the data. This method of improving error rate at the ex- 
pense of data rejection for algebraic codes is not unique. Consequently the amount of improvement 
shown is not necessarily optimum. 

In the process of obtaining the curves of Figure 2, r and E b were allowed to vary in each case 
to values that produced a constant deletion rate, D. This was done in order to examine the varia- 
tion of the word error probability as a function of r and E b . Observation of the Tables indicates 
that the variation of P w is insignificant, although it reaches some minimum which corresponds to 
higher r f s as the signal to noise ratio increases. Since higher values of r correspond to higher 
threshold values, the above result is reasonable. 


*See appendix. 

tThere is a extra term which has been ignored because it is very small for the cases considered here (see Appendix). 
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Examination of Figure 4 shows that 


(a) The performance of orthogonal and biorthogonal codes is indistinguishable for all curves 
(d = 0, .03, .05, ,3). 

(b) At 3 db the orthogonal and biorthogonal codes are superior to the convolutional codes for 
D = .3. 

(c) At 3.5 db the orthogonal and biorthogonal codes are slightly better than the convolutional 
qodes for D = .05. 

(d) At 4 db, D = .03 the performance of the two different systems is about the same. 

(e) The improvement in performance as D increases is much more pronounced in orthogonal 
and biorthogonal codes than in algebraic codes. 

Figures 5 to 9 show the performance of orthogonal and biorthogonal codes for some other n's. It 
should be noticed that for relatively larger n orthogonal and biorthogonal codes perform equally 
well. 
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APPENDIX A 


1. In the case of N biorthogonal signals the number of correlators is N/2 * and the probability of 
correct decision is the probability that the useful signal is larger than the absolute value of any of 
the (N/2) - 1 remaining noise signals and larger than B t , given that the sign of the useful signal has 
been correctly determined; thus 

P c - prob (| s| > | x | , Vx; |s| >B t ) 

or equivalently 

P c = prob (| s| > |x (L) |; |s|>|x (1) |; |s|>B t ). (A-l) 

This statement is equivalent to the probability that the largest and smallest noise signals are within 
the range (-s , s); for Gaussian signals with zero mean 

prob (| s| > |x (L) |) = prob (| sj > |x (i) |j . (A-2) 

Thus, independently of the relative frequency of positive and negative useful signals. 

P c = prob (| s | >| x (L) | ; |s|>B t ) 

- <N/2)-l 

f (x) dx (A-3) 

B t > 0 . 

2. An error occurs whenever the absolute value of either the smallest or the largest noise signal 
is larger than the absolute value of the useful signal, or the useful signal is mistaken for its nega- 
tive. If the above mentioned events are named events A, B and C respectively, then, 

P e = P(AUBUC) = P(A) + P(B) + P(C) - P(AflB) 

- P(BflC) - P(AflC) + P(AflBnC) . (A-4) 

*N = 2 n is the number of correlators necessary for N orthogonal signals. 
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Since A, B, c are independent 


P e = P(A) + P(B) 

+ P(C) - P(A) 

P(B) 

- P( A) P(C) - P(B) P(C) 





+ P( A) P(B) P(C) . 

(A-5) 

and P(C) are given by 





P(A) 

= P(l* (1) l > 1 

Is|; | 

x ( l)l >B t) 

(A-6) 

P(B) 

P (1 X (N) 1 > 

M; 1 

X (N) 1 > B t) 

(A-7) 

P(C) 

= P(|-s| >X, 

Vx; | 

-s| >B t ) . 



P(B)is given by 


r Tr T /2> ’ 2 r 

i) = J (? - l) f(*) j f(t)dt j 
B t L'“ J 


f. (s)ds 


Again due to symmetries in f (x (1) )and f(x (N) ) , and symmetry of f s (s)for the word and its 
complement, 


Finally 


P(A) « P(B) 


P(C) = J f s (~s)ds J f(x)dx 


f s (~s) = 


1 e -(stA) 2 /2 


and P(C) has a negligible value for values of s > o (B t > 0) and can be neglected for A >3 and n > 16 in 
this paper. This can be seen in Figure A1 where f (-s) and 


f(t)dt 


16 


have been plotted. 


where 


F l (|s I) = (2F(s)- l)^ 72 )' 1 


F(s) 



f(t)dt . 


Substituting these results into (A- 5) with P(C) * 0 


P e = P(A) + P(B) - P(A) P(B) 


= a( 2 - a) . 
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LIST OF SYMBOLS 


n the word length 

N the number of words 

k number of information bits per word 

e the maximum bit correction capability of a code 
E u the bit erasure rate 

D the word deletion rate 

p the bit error probability 

Q the probability of correct detection (on a bit basis) 

E p the probability that an erasure bit is in error 

E q the probability that an erasure bit is correct (E p + E q = E b ) 

p M l d s error rate of a Maximum Likelihood Decision System 

Q MLDS 11 1 " p mlds J the probability of correct detection for MLDS 
P c probability of correct detection (on a word basis) 

P w , P e word error rate 

E the signal energy per bit of information 
N 0 noise power per unit bandwidth 

P correlation coefficient 

U the union of events 

n the intersection of events 

V for all 

x (L) the largest of the noise signals. 
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Figure A1 




